The Best Way to Start Reinforcement Learning

Recently I gained interest in Reinforcement Learning (RL) for robotics, especially after working on AI orchestration during my internship. After trying a few things, I found that the best way to start is to use the Gymnasium library for creating environments and the Stable Baselines3 library for training RL agents.

These two open-source libraries are perfectly suited for beginners in RL as you can quickly get some results without having to implement everything from scratch and then implement your own environments and agents as you progress. But the best part is that they have out-of-the-box support for robotics simulation environments in Mujoco which is exactly what I was looking for.

If you want to try it out, here is a dependency list for you favorite package manager:

dependencies = [
    "stable-baselines3[extra] (>=2.6.0,<3.0.0)",
    "gymnasium[other] (>=0.29.1,<1.2.0)",
    "box2d (>=2.3.10,<3.0.0)",
    "swig (>=4.3.1,<5.0.0)",
    "gymnasium-robotics (>=1.3.1,<2.0.0)",
    "sb3-contrib (>=2.6.0,<3.0.0)",
]

If this didn't convince you yet, here is what I achieved after a few tries with the Double Pendulum environment and the CrossQ algorithm :

This is a simple example, but as someone who is used to working with classical control methods, I found it very interesting to see how easy it is to get a working RL agent that can control a complex system like this one.